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ABSTRACT 

This article explores the following questions: first, 
*'Do colleges of higher vocational education vary in the mean success 
of their graduates on the labor market?** and, second, **Are the 
differences between these colleges stable over time?**. The analysis 
drew upon school effectiveness, quality assurance and labor market 
research. The school effectiveness literature shows that schools 
differ in the educational output of their students. The relationship 
between the organizational context and structure and the paradigms of 
these colleges affect the quality of the outcomes of educational 
policy. The leading assumption in human capital theory is that since 
education makes people more productive, education is an investment in 
human capital. An annual survey of graduates of colleges in the 
Netherlands collects information about destination, labor market 
entry, unemployment spells and the jobs held by graduates. 

Statistical analysis showed: (1) systematic differences between 
departments and colleges in the labor market outcomes of their 
graduates even after controlling for student and regional factors, 

(2) stable department effects with respect to the length of 
unemployment and job level; (3) difference between the net and gross 
quality indicators of a department; and (4) no correlation between a 
department *s improving graduates* chances of finding work and the 
quality of work. (Contains 33 references.) (CK) 
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1 introduction 



In most western countries, a growing interest in educational indicators can be demon- 
strated. These indicators are used as summary statistics for measuring the quality of the 
educational system. The use of key indicators enables policy makers and educational 
authorities to assess the performance of their educational system, by enabling a 
systematic comparison of the quality of education between countries, between years, 
between regions, between institutes or between types of education. 

This increase of interest in the public reporting and accountability of education is due to 
the following factors: 

* a growing awareness that improvement of the quality of education is necessary to 
increase Europe's competitiveness on a global market and to face the challenges of 
the future (c.f. White Paper, 1993; IRDAC, 1990); 

* constraints on educational budgets due to higher enrollments, forcing the institutes of 
education to perform more effectively and efficiently; 

* increasing demands from consumers (parents and students) to be informed about the 
quality of educational institutes. 

On an international level, the interest in educational indicators can be demonstrated in 
projects like the indicators of Education Systems (INES) project or the Community 
Educational Indicators Project (CEIP) from the European Commission (West et al, 1995). 
The EC interest in quality of education is also demonstrated in the Maastricht Treaty on 
European Union: 'EC shall contribute to the development of quality education....' (article 
126, chapter 3) and in subsequent action programmes like SOCRATES and LEONARDO. 
On a national level, similar projects can be identified, regarding the educational perfor- 
mance on a national level as well as a regional and institutional level (National Council for 
Education and Training Targets, 1993; Willms and Kerckhoff, 1995; Croxford and Cowie, 
1 996; Bosker and Scheerens, 1 995; Ministry of Education and Science, 1 995). 

In the Netherlands, quality assessment in higher education has been on the public policy 
agenda since the publication in 1985 of the paper "Higher Education: Autonomy and 
Quality" by the Dutch Ministry of Education and Science’. This paper marked a change in 
the administrative relation towards higher education. Instead of a direct control of the 
contents and quality of higher education, the Ministry placed higher emphasis on 
autonomy and self regulation for the institutes of higher education (colleges for higher 
vocational education and universities). This was to be accompanied by a system of 
quality assurance in which the institutes of higher education give an account of their 
performance. 



1 . Ministry of Education and Science (1 985), Hoger Onderwijs: Autonomie en Kwaliteit (HOAK-nota). 



This system of quality assurance covers all areas of the educational process, such as 
inflow of students, study progress (internal output), input of staff and other resources, 
but also business administrative aspects. One of the major issues in the system of 
quality assurance is the so-called external output of education, that is the success of 
graduates on the labour market. 

One of the basic assumptions of the system of quality assurance is that colleges may 
differ in their quality. Moreover, one assumes that these differences are caused by 
■ organisational or educational factors, rather than differences in student composition or 
regional differences in the labour market, factors for which the colleges cannot be held 
responsible. Furthermore one assumes that these differences are systematic and not 
subject to major variations over time. 

In this article we explore the following questions: 

1. Do colleges of higher vocational education vary in the mean success of 
their graduates on the labour market, even after controlling for personal 
and regional characteristics? 

2. Are the differences between these colleges stable over time? 



2 Theoretical background 

The analysis in this article draws upon three lines of research: school effectiveness, 
quality assurance and labour market research. 

The school effectiveness literature shows that schools differ in the educational output of 
their students even after controlling for input factors like intelligence, motivation and 
socio-economic background. These school effects can be attributed to either school, 
classroom or teacher factors and explain between 9 and 27% of the variance in educati- 
onal attainment (Mortimore et al., 1988; Brandsma and Knuver, 1989). The effects seem 
to be larger in primary education than in secondary education (Bosker, 1990). The 
theoretical models concentrate either on the process of learning and effective instruction 
(e.g. Creemers, 1992 and 1994) or on the school as an organisation (e.g. Scheerens, 
1990). The former puts a heavy emphasis on learning time and opportunity to learn, 
while the latter emphasises organisational features like strong educational, leadership, 
achievement oriented policy and an orderly and safe climate. 

In the leading reviews of the school effectiveness research (e.g. Creemers and Schee- 
rens, 1994; Scheerens, 1993) no mention is made of research into the labour market 
outcomes. Most of the research concentrates on academic perforrrance or school ca- 
reers. This, of course, is caused by the fact that most research concentrates on primary 
education. However, one might argue that differences in the educational output will also 
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be reflected in differences in labour market outcomes. The school effects in labour 
market outcomes however will probably be less strong, because of the intervening 
effects of other factors. 

The second line of research which bears relevance for our analysis relates to research 
into quality assurance and the use of performance indicators in higher education. Quality 
assurance, planning and resource allocation, particularly during times of financial 
(governmental) constraints and changing socio-economic demands, involves making 
choices between mutually exclusive alternatives each with its own combinations of 
inputs, outputs, outcomes, impacts and benefits (e.g. Segers, 1993). At the one hand 
the institutions of higher education obtained a greater autonomy with respect to the 
'input' and 'process' factors. On the other hand the institutions are being called more 
and more to account for their 'output'. Effectiveness and efficiency became central 
concepts: higher education became responsible to 'manage' their institutions by themsel- 
ves. In this framework it is relevant to mention that De Jager (1994) shows that the 
relationship between the organizational context and structure, and the 'paradigms' of the 
higher vocational colleges have an impact on the outcomes of educational policy. These 
paradigms refer to the pedagogical or professional role of the staff, the educational 
mission and internal coalition of the college. 

Countries like the United Kingdom, Australia and the Netherlands adopted ideas and tools 
of the American 'accredition system'. In the Netherlands this lead to a nation wide and 
public quality assurance system of 'visiting committees'. All institutions taking care of a 
discipline are 'visited' periodically. It is a rolling review process with two complementary 
phases: internal evaluation by the faculty, reported in a selfstudy report and visitations 
by external commentators. A checklist, consisting of a set of performance indicators, the 
operationalization of the dynamic and multidimensional concept of 'quality', is the 
guideline for the faculty report and the comments of the visiting committee. The 
relevance of the training with respect to the requirements of the labour market and 
professional practice are more and more considered as important indicators to judge the 
quality of a higher (vocational) educational discipline. 

The third line of research of relevance here is the labour market research. The leading 
assumption in human capital and matching theory is that education and training makes 
people more productive. Following education can therefore be seen as an investment in 
human capital (Becker, 1975; Schultz, 1961) which increases productive skills and will 
therefore lead to higher v/ages. In most research the focus of interest has been on 
explaining differences between types of education, rather than between institutes of 
education. A few authors point to the relevance of differences between colleges. 
Glebbeek (1989) regards education as a production process with an important feature: 
the variability in outcomes. Unlike most production processes, the output of education is 
characterised by a large variation in quality. Graduates vary in considerably in acquired 
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skills and this variation is closely linked to the selectivity of the educational process. 
Colleges that put high standards in their exams, will produce an output with higher 
quality and less variation. As employers have no means of assessing directly the 
productive skills of an individual, this variation in quality produces a considerable risk. 
Especially in branches in which mediocre functioning of employees produces a high risk 
(like health care), this could lead to additional educational requirements. Empirical 
evidence for this relation between the quality of a college and overeducation is given by 
Robst (1995). In a paper examining the relation between college quality and overeducati- 
on, he shows that workers who attended a lower quality college have a higher likelihood 
of ending up in a job for which they are overeducated. In the Netherlands Van der Velden 
et al. (1989) has showed that the following factors are relevant for the labour market 
outcomes: high selectivity during the training, an appropriate match between education 
and occupation, effective instruction and an active labour market policy. 

In this article we restict the analysis to the question whether there are any differences 
between colleges of higher vocational education after controlling for student and regional 
factors and leave aside the question to which factors these differences may be attribu- 
ted. Bearing in mind the results of the school effectiveness literature, we expect only 
modest differences between colleges in the labour market outcomes. Moreover, we 
expect these differences to be larger for employment chances than for outcomes which 
are more institutionally defined (e.g. wages). 



3 Data and method 

In 1991, the Council for Higher Vocational Education (HBO-Raad) commissioned the 
Research Centre for Education and the Labour Maket (ROA) to develop an instrument 
which assesses the success of graduates on the labour market and which could be used 
in the system of quality control by institutes of higher vocational education. This 
instrument has been in operation as of 1991. About two thirds of all the colleges make 
use of this instrument, which makes it the most Important instrument for the external 
quality control. 

The instrument consists of an annual survey among graduates of the participating 
colleges. In this survey, information is collected about destination, labour market entry, 
unemployment spells and the jobs that the graduates hold. The survey is held one and a 
half year after graduation. 

The participating institutes of higher vocational education receive standard information 
about the results of their 'own' graduates in confidential institute reports. This informati- 
on can directly be compared with the national figures for the types of education in 
question. Apart from this, a national, public report is prepared where the results of the 



total outflow of a particular sector of higher vocational education is analysed (see Van de 
Loo and Van der Velden, 1 994) 

In this analysis, we use data from the 1991, 1992 and 1993 survey (Van de Loo et. al., 
1 992; Van de Loo et. al., 1 993; Van de Loo and Van der Velden, 1 994). This means that 
the research population consists of graduates from higher vocational education from the 
following cohorts: 1989/90, 1990/91 and 1991/92. We restrict the dataset to graduates 
from the largest types of training within the following 6 sectors of higher vocational 
education: pedagogics, technics, economics, health studies, social studies and fine arts. 
The selected courses are: teacher training (for primary education), mechanical enginee- 
ring, business administration, nurse training, social work training and music (teacher) 
training. 

In the analysis we used the following dependent variables: a dummy indicating if the 
graduate was less than 3 months unemployed during the job entry period ("job entry”), a 
dummy indicating if the level of the job correponded to the level of higher education 
("level of job")^ and two indicators for income ("gross hourly wage" and "gross monthly 
wage"). 

The predictor variables include personal characteristics (sex, age) as well as qualification 
and skills characteristics like training course in higher vocational education, previous type 
of education, previous job experience, board experience and length of study. To control 
for regional and annual (business cycle) variations we include the region where the 
college was situated, the region of work and year of survey. 

Table 1 presents some descriptive statistics. Some 15% of the graduates were more 
than 3 months unemployed during the period between graduation and the survey. At the 
moment of the survey 79% held a job which matched their educational level. The gross 
monthly wages amounted to 2.880 Dutch guilders, while the gross hourly wage was 
slightly more than 18 guilders on average. 

A majority (60%) of the sample used for this analysis is female. The mean age is 25 
years. About a quarter of the graduates had board experience before entering the labour 
market and about one third had previous job experience. Senior general secondary 
education is the dominant type of previous education, followed by pre-university 
education and senior vocational secondary education^. 



2. Zero corresponding with a job level lower than higher education (overeducation). 

3. Senior general secondary education (HA VO) has a duration of 5 years after primary education and is 
supposed to be the main route to higher vocational education. Pre-university education (VWO: duration 6 
years) is supposed to prepare for university education, although an important share of its graduates go on 
to higher vocational education. Senior vocational secondary education (MBO duration 3-4 years) prepares 
mainly for the labour market although it has an increasing share of graduates following further education. 
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Table 1 

Means (standard deviations) and proportiorts 





job entry 


income 
& level of job 


Dependant variables 


job entry 




n.a 


- quick 3 months) 


85.3% 




- slow 


14.7% 




level of job 


n.a. 




- lower than higher vational 




20.9% 


' at least higher vocational 




79.1% 


wages per hour (fl) 


n.a. 


18.29 (4.23) 


monthly wages (kfl) 


n.a. 


2.88 (0.66) 


Predictor Variables 


Student level 


N.= 6275 


N.= 4303 


sex 


- female 


60.0% 


62.0% 


- male 


40.0% 


37.8% 


preliminary education 


- senior general 


46.5% 


45.5% 


- pre-university 


31.1% 


31.3% 


- senior vocational 


19.6% 


20.6% 


- higher vocational 


0.9% 


0.7% 


- other education 


1 .9% 


2.0% 


job experience 


- yes 


33.8% 


34.0% 


- no 


66.2% 


66.0% 


board experience 


- yes 


25.2% 


25.2% 


- no 


74.8% 


74.8% 


age 


24.63 (2.05) 


24.70 (2.06) 


length of study (in years) 


4.05 (0.63) 


4.03 (0.62) 


region of work 


n.a. 




- west 




48.7% 


- north 




6.5% 


- east 




20.7% 


- south 




22.5% 


- foreign 




1 .6% 
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Table 1 (contin< lation) 

Means (standard deviations) and proportions 





job entry 


income 
& level of job 


year level 


N^ = 223 


N^ = 223 


year 


- 1991 


28.7% 


28.7% 


- 1992 


34.1% 


34.1% 


- 1993 


37.2% 


37.2% 


deoartment level 


N, = 97 


II 

<o 


study 


- pedagogics 


25.8% 


25.8% 


- technics 


18.6% 


18.6% 


- economics 


15.5% 


15.5% 


- health studies 


17.5% 


17.5% 


- social studies 


16.5% 


16.5% 


- fine arts 


6.2% 


6.2% 


collsge level 


N. = 50 


N. = 50 


region 


- west 


42.0% 


n.a. 


- north 


10 0% 




- east 


26 0% 




- south 


22.0% 





The number of graduates of higher vocational education is (still) increasing. This is not 
only the case in this analysis but also in the total amount of this population in the 
Netherlands. The western part of the Netherlands is both dominant in the educational 
field, and in the employment. For the northern and eastern regions there is a negative 
nett migration flow of graduates to this dominant western part in the Netherlands. 

A nested design is used consisting of the following levels: student, year, department 
(sector of college) and the college level. As we selected only one course per sector, the 
department level coincides with the training course. We restricted the dataset to those 
graduates who, at the moment of the survey, belong to the labour force (employed plus 
unemployed). This amounts to a total of 6.275 cases'*. These were nested in 223 
year/department/college combinations. These combinations were nested in 97 
department/college combinations and these were again nested in 50 different colleges. 



4. For the analysis of job entry the total sample could be used. For the other three analyses the sample was 
restricted to those graduates who held a job at the moment of the survey. This amounts to 4.303 cases. 




n 
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The model specification is as follows: 



(1) V.ydc = Boydc + + ... + Si 2 FOREIGN.yd, + e.ydo student-level 

(2) Bovdo = Boodo + Yi 1992yd, + Y2l993ydo + Uoydc year-level 

(3) l^oodc “ ®oooc + SiTECHdc + ... + SsARTSj, + Voodc department-level 

(4) Boooo = Soooo + NORTH, ... ^aSOUTH, Woooo college-level 

Furthermore it is assumed that e,yd, "N(0,o^), Uoydc ”N(0,Ou), Voodc ”N(0,Ov), and that 
Woooc ”N(0,a^>, i.e. all residuals are normally distributed with zero mean and some 
(unknown) variance. By definition the residuals are uncorrelated. Regression and variance 
component parameters are estimated using maximum likelihood. 

For the logistic multi-level regression model, applied to model the binary dependent 
variables (job entry and level of job) the logistic link function is used (Wong & Mason, 
1985). In this case it is assumed that 

(5) (y,vdc I P.vdc) " Bernouilli(p,ydc) 



where p,vdc = PR(V.vdc = 1 ) 's the probability that student s from year y in department d of 
college c will have a job respectively will have an appropriate job. Instead of y,y<j, 
logit(p,yd,) will be regressed on the predictors. In this case (1) does not contain a residual, 
i.e. e,yd, = 0. Parameters are estimated using quasi-likelihood estimation (Longford, 1993). 

The stability of the performance of the departments is estimated by taking the ratio of 
the between years variance to the sum of the between years and between departments 
variance: 

stability: o„ / (o„ + Ov 

Since the random effects are modelled explicitly, this is the 'true' stability. 

Tie performance indicators than are constructed using 'shrinkage estimators' (e.g. Bryk 
& Raudenbush, 1992), in which precision weights are used to estimate the department 
param'^jters: 

B‘d, = Wd,Bj, + (1-Wd,)B* 
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B is the Ordinary Least Squares estimator (in models without predictors this is the mean), 
Wdi the precision weight and 6* the Empirical Bayes estimate of the population mean 
(across the departments and colleges). These precision weights are the ratio's of the true 
between department level variance relative to the observed between department level 
variance; 

Wde = / (o^ + (al + 



nydc is the harmc'C mean of the number of students in year y in department d in college c 
and n<jg is the number of years in department d in college c. 

The gross performance indicators are constructed estimating the parameters without 
predictors. The nett performance indicators are constructed estimating parameters in 
regression mod<3ls that contain all predictors with statistically significant effects. 



4 The analysis 

As indicated before, we used a 4-level hierarchical model for the analysis with levels 
student, year, department and college. For the dummy variables job entry and level of 
job we adopted a logistic regression approach (see Bosker and Hofman, 1994 for an 
elaboration). Table 2 presents the results of the 4-level analyses with predictors year of 
survey and type of training only. The residual outcomes per department of this analysis 
equates the mean of the 'gross' performance indicators as presented in the so-called 
confidential institute reports in those three years. The rankings of the departments on 
the basis of these mean gross outcomes for their graduates are presented in the 
appendix. 

Compared to the situation in 1991 the labour market opportunities for graduates 
worsened in 1992 and 1993, with significantly lower proportions of graduates who were 
less than 3 months unemployed. This was not accompanied by a significantly lower level 
of the jobs acquired, although the negative sign suggests that the proportion of gradua- 
tes who hold a job below their educational level has grown. The hourly and monthly 
wages in 1992 and 1993 are significantly above the 1991 level. Between 1992 and 
1993 there is a drop in the mean monthly wages, which may be attributed to an increase 
in parttime work and lower level jobs. 

There are significant differences between the types of training in labour market outco- 
mes. Compared to 'teacher primary education', graduates from 'nurse training' show sig- 
nificantly higher proportions of quick job entry while the graduates from 'social services' 
and 'music teacher' suffer significantly more from unemployment. 
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Table 2 

Results of the (logistic) 4-level analyses. Regressions coefficients and standard errors (between 
brackets) 





job entry 
(quick) 


level of job 
(high) 


wages per 
hour 


monthly wages 


GRAND MEAN 


1.840 


2.400 


15.925 


2.481 


year level 
year 

- 1991 
- 1992 

- 1993 


0.000 
-0.247 (.113) ■ 
-0.329 (.112)" 


0.000 

-0.100 (.104) 
-0.176 (.102) 


0.000 

1.080 (.178)" 
0.998 (.174)" 


0.000 

0.120 (.028)** 
0.056 (.028)* 


department level 
study 

- pedagogics 

- technics 

- economics 

- health studies 

- social studies 

- fine arts 


0.000 

0.234 (.178) 
0.018 (.182) 
1.253 (.205)" 
-0.515 (.200)' 
-1.275 (.267)" 


0.000 

-0.579 (.167)" 
-1.136 (.154)" 
-1.519 (.150)" 
-1.437 (.187)" 
-1.217 (.315)" 


0.000 

2.570 (.212)" 
2.204 (.200)" 
1.677 (.199)" 
2.846 (.281)" 
5.542 (.506)" 


0.000 
0.689 (.035)** 
0.609 (.035)*' 
0.256 (.034)** 
0.126 (.046)** 
-0.540 (.081)*' 


*: p < .05 ■*: p 


< .01 








The analysis with respect to level of job shows the typical features of the labour market 
for teachers, where job entry is ruled by specific demands regarding the proper qualifica- 
tions and where the graduates almost without exclusion become teacher. The hourly 
wages for graduates from 'teacher primary education' are the lowest of all 6 training 
courses and the same applies for the monthly wages, with one exception: the graduates 
from 'music teacher' have lower monthly wages because of a larger proportion of people 
holding parttime jobs. 


Table 3 

Results of the (logistic) 4-level analyses for 'gross' performance indicators 
Residual variance components 






job entry 


level of job 


wages per hour monthly wages 


student level 
year level 
department 
college level 


n.a. 

0.069' 

0.103' 

0.105* 


n.a. 

0.000 

0.059* 

0.000 


16.067* 

0.180* 

0.000 

0.000 


0.376* 

0.005* 

0.000 

0.002* 



■; p < .01 
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Table 3 shows the residual variance components at the different levels.^ For the analysis 
on job entry, we see that there are significant differences in the mean outcomes at all 3 
levels. Of the total residual variance of 0.277 (=0.069 + 0.103 + 0.105) at higher 
levels, 25% can be attributed to differences within departments between the years, 37% 
to differences between the different departments within a college and the resulting 38% 
to differences between colleges. In other words, from the total amount of differences 
between departments and colleges over the years, 25% is due to change, or put 
alternatively: the stability of department and college effects for job entry amounts to 
75%. 

For the analysis on level of job, we only detect systematic differences between depart- 
ments, with no variation over time and no differences between colleges. 

For the analysis on hourly wages the picture is quite different. The total variance 
amounts to 16.247, only 1% of which can be attributed to differences between depart- 
ments over the years. The stability of these department effects is zero: ail differences 
found are due to variation in time. In the analysis on monthly wages only 1.8% can be 
attributed to differences between departments, most of which again is due to variation 
over time (stability: 28%). 

Table 4 presents the results of the (logistic) 4-level analyses after controlling for student 
and regional factors. The results regarding the type of training followed and the year of 
survey, are very similar to the former results in table 2. Regarding the effects of personal 
characteristics, we note that males earn 0.4 guilders per hour more than females. Per 
month the difference amounts to 134 Dutch guilders in favour of males. The type of 
previous education followed, only has an effect on the job entry: graduates who have 
followed senior vocational secondary education before theii course, in higher vocational 
education, have a higher chance of being less than 3 months unemployed. Having job 
experiences before graduation increases the employment chances and also increases the 
monthly wages by 51 guilders. There is no significant effect of board experience. Age 
decreases the chances of being less than 3 months unemployed, but once a job is found 
it also increases the wages earned. Graduates who finish their study in a relatively short 
time, increase their chances of being less than 3 months unemployed and also increase 
their chances of acquiring a job which matches their educational level. Finally, regional 
differences in the labour market situation affect the labour market outcomes. Graduates 
who attended college in the western part of the Netherlands, have significantly higher 
chances of being less than 3 months unemployed than those who attended college in the 
south, the east and especially the north. Moreover, wages in the west are significantly 
higher than in the other parts of the Netherlands, amounting to 164 guilders difference 



5. At the student level there is no variance component for the logistic analyses, because the dependent 
variable is a dummy. 




15 



11 



with the mean monthly wages in the north. Those who found a job abroad earn the 
highest wages with a difference of 677 guilders compared to the monthly wages in the 
west. 



Table 4 

Results of the (logistic) 4-level analyses fc' 'nett' performance indicators. 

Regressions coefficients and standard errors (between brackets); only significant (pi. 05) coeffi- 
cients are shown 





job entry 
(quick) 


level 
of job 
(high) 


wages 
per hour 


monthly 

wages 


GRAND MEAN 


5.330 


3.386 


8.025 


1.741 


Student level 










sex 

- female 

- male 






0.000 

0.375 (.168) 


0.000 
0.134 (.026) 


preliminary education 

- higher general 

- pre-university 

- senior vocational 

- higher vocational 

- other educatation 


0.000 
-0.011 (.096) 
0.219 (.116) 
-0.170 (.352) 
0.108 (.259) 








job experience 

- no 

- yes 


0.000 

0.208 (.088) 






0.000 
0.051 (.021) 


board experience 

- no 

- yes 










region of work 

- west 

- north 

- east 

- south 

- foreign 


n.a. 




0.000 

-0.377 (.261) 
-0.354 (.165) 
-0.323 (.163) 
4.272 (.481) 


0.000 
-0.164 (.044) 
-0.117 (.028) 
-0.082 (.028) 
0.677 (.074) 


age 


-0.103 (.021) 




0.332 (.032) 


0.031 (.005) 


length of study 


-0.148 (.065) 


-0.241 (.067) 
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Table 4 (continuation) 

Results of the (logistic) 4-level analyses for 'nett' performance indicators. 

Regressions coefficients and standard errors (within brackets), only significant (p < .05) coeffi- 
cients are shown 





job entry 
(quick) 


level 
of job 
(high) 


wages 
per hour 


monthly 

wages 


year level 










year 


- 1991 


0.000 


0.000 


0.000 


0.000 


- 1992 


-0.274 (.113) 


-0.116 (.104) 


1.099 (.170) 


0.122 (.027) 


- 1993 

department level 


-0.328 (.111) 


-0.184 (.102) 


0.975 (.165) 


0.055 (.027) 


Study 


- pedagogics 


0.000 


0.000 


0.000 


0.000 


- technics 


0.224 (.183) 


-0.474 (.170) 


1 .873 (.242) 


0.548 (.040) 


- economics 


0.000 (.178) 


-1.097 (.155) 


1.762 (.201) 


0.521 (.036) 


- health studies 


1 .262 (.209) 


-1.515 (.150) 


1.416 (.189) 


0.210 (.034) 


- social studies 


-0.568 (.200) 


-1.471 (.187) 


2.332 (.276) 


0.072 (.045) 


- fine arts 


-1.045 (.279) 


-0.815 (.336) 


3.594 (.507) 


-0.823 (.082) 


college level 


region 

- west 

- north 

- east 

- south 


0.000 
-0.937 (.204) 
-0.534 (.153) 
-0.562 (.153) 


n.a. 


n.a. 


n.a. 



Table 5 presents the residual variance components at the different levels, after taking 
account of the student and regional factors. We note that the residual variance at the 
college level for 'joh entry' has dropped considerably. The total residual variance now 
amounts to 0.193, which means that 30% of the original residual variance at the three 
higher levels is explained by student factors or region. There is no residual variance left 
at the college level, indicating that differences between institutes of higher vocational 
education must be attributed to the effects of a department, rather than the effect of the 
college as a whole. To give an example, a college may have a relatively well performing 
technics department, and at the same time a relatively bad performing health depart- 
ment. The performance of the different departments within a college is unrelated to each 
other. A part of the department effects varies over the years. The stability of the depart- 
ment effects, after controlling for student and regional factors, can now be estimated to 
be 68% (=0.132/(0.061 +0.132)). 
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Table 5 

Results of the (logistic) 4-level analyses for 'nett' performance indicators 
Residual variance components 





job entry 


level of job 


wages per hour 


monthly wages 


student level 


n.a. 


n.a. 


15.253* 


0.356* 


year level 


0.061 * 


0.000 


0.129* 


0.004* 


department level 


0.132* 


0.058* 


0.000 


0.000* ” 


college level 


0.000 


0.000 


0.000 


0.002* 



*: p < .01 

” 0.0002 to be precise, and significant at a < .01 



For 'level of job' we note results very similar to the former model. There is no residual 
variance at year or college level. Introducing student and regional variables caused only a 
very small drop in the residual variance at the department level. Only 2% of the original 
residual variance at this level is 'explained' by these variables. 



For 'hourly wages' and 'monthly wages' the model hardly improved by introducing the 
student and regional variables. In the case of 'hourly wages' the drop in residual varian- 
ces at all levels is only 5%. The residual variances at department and college level remain 
zero: there are no systematic differences between departments or colleges in this 
respect. In the case of 'monthly wages', introduction of the student and regional 
variables 'explains' 6% of the original residual variance. There remain very small 
systematic differences in labour market outcomes between colleges and departments, 
but a large part of these effects is due to variation over time. 

The residual effects at the department level may be regarded as the performance indica- 
tor of a department regarding the labour market outcomes. In this respect, it is interes- 
ting to compare the 'gross' performance with the 'nett' performance, that is the perfor- 
mance after taking account of the student and regional factors. Table 6 presents the 
correlations between the gross and nett performance indicators. One conclusion that 
may be drawn, is that the correlation between gross and nett indicators is high, but not 
perfect. In the case of 'job entry' the correlation between the two indicators is .77, 
indicating that only 59% of the differences in the nett outcomes of a department is 
correctly predicted by the gross outcomes. In the case of 'monthly wages' the predictive 
value of the gross outcomes is 67%, while in the case of 'level of job' the predictive 
value of gross outcomes is almost perfect. This can also be illustrated by the rankings 
presented in the appendix. In the case of 'job entry' in 10 out of 97 cases the ranking 
of gross and nett outcomes differs more than 40 places, turning a relatively well- 
performing department into a relatively bad one and vice versa. In the case of 'monthly 
wages' this happens only in 5 out of 97 cases, while in the case of 'level of job' it does 
not happen at all. 
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5 Summary and conclusions 



The main results can be summarized as follows: 

* There are systematic differences between departments and colleges in the following 
quality indicators with respect to the labour market outcomes of their graduates: 
chance of being less than 3 months unemployed; chance of acquiring a job which 
matches the educational level attained and gross monthly wages. 

* These effects remain significant even after controlling for student and regional factors. 
They are due to differences between departments rather than differences between 
colleges. 

* The department effects are rather stable in the case of chances of being less than 3 
months unemployed and of acquiring a job which matches the educational level 
attained. In the case of monthly wages the effects vary considerably over time. 

* The nett quality indicator of a department (i.e. the outcome after controlling for 
student and regional factors) differs considerably from the gross quality indicator, 
except in the case of level of job. 

* Departments which effectively improve their graduates' chances of finding a job are 
not necessarily also effective in improving the quality of work, as indicated by level of 
job or wage. 

What are the theoretical and empirical implications of the previous analysis? 

First of all, it seems important to extend school effectiveness research to the area of 
labour market outcomes. Colleges do have an effect on the labour market success of 
their graduates. The leading theories within the school effectiveness research, however, 
seem to bear only little relevance with regard to these labour market outcomes. To build 
a coherent theoretical framework, theories of effective instruction and effective schools 
need to be integrated with theories about the role of education in selection processes on 
the labour market (c.f. Van der Velden et.al., 1989). 

The analysis shows that schools which are effective in the area of improving their 
graduates' employment chances, are not necessarily effective in the area of improving 
the quality of the jobs which the graduates hold. This is in line with previous analyses of 
Van der Velden and Wieling (1994) and Van Smoorenburg and Van der Velden (1995), 
who showed that the labour market position of types of education can be distinguished 
in two major dimensions: employment chances and quality of work. These dimensions 
can be shown to be theoretically and empirically independent. The fact that these two 
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dimensions are also relevant in the analysis of differences between institutes, stresses 
the importance of this distinction. In the theoretical framework it is important to 
elaborate on the factors which makes an institute more effective in one area and which 
factors are of importance in the other area. 

Bearing in mind the growing importance of external quality control both for higher 
education as well as for secondary education, implicates that there is an increasing need 
for valid and reliable information about the performance with respect to labour market 
outcomes of institutes of education. In this respect, the added value of an institute, i.e. 
the performance after controlling for input or regional factors, is of particular interest. 
Given the fact that the nett performance of an institute quite often differs considerably 
from the gross performance, institutes should only be judged on the basis of this nett 
performance. The model presented here offers an adequate way of calculating these 
'nett' quality indicators. 
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Appendix 



Bankings of studies and colleges on 8 performance indicators" 



Study 


cono 


jeg 


jen 


jig 


jin 


whg 


whn 


mwg 


mwn 


2 


58 


22 


18 


30 


28 


51 


51 


2 


3 


2 


28 


26 


80 


26 


26 


51 


51 


9 


40 


2 


43 


10 


8 


22 


22 


51 


61 


12 


25 


2 


79 


24 


19 


18 


18 


51 


51 


21 


20 


2 


70 


80 


67 


67 


69 


51 


51 


27 
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80 


74 


28 


39 


38 


51 


51 


28 
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17 


45 


65 


35 


36 


51 


51 


34 


48 
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83 


46 
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96 
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53 
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Rankings of studies and colleges on 8 performance indicators (continuation)'* 



study 


cono 


jeg 


jen 


i<9 


jilt 


whg 


whn 


mwg 


mwn 


4 


51 


12 


30 


65 


67 


51 


51 


4 


12 


4 


28 


11 


21 


31 


30 


51 


51 


10 


44 


4 


7 


67 


25 


2 


2 


51 


51 


13 


6 


4 


30 


25 


37 


99 


100 


51 


51 


19 


45 


4 


80 


97 


96 


49 


52 


51 


51 


31 


13 


4 


17 


44 


60 


58 


55 


51 


51 


34 


51 


4 


13 


62 


75 


4?. 


43 


51 


51 


36 


38 


4 


73 


87 


69 


19 


19 


51 


51 


40 


15 


4 


63 


60 


31 


■'2 


72 


51 


51 


42 


31 


4 


37 


3 


12 


45 


44 


51 


51 


43 


74 


4 


64 


77 


94 


98 


96 


51 


51 


46 


65 


4 


67 


5 


9 


10 


12 


51 


51 


51 


46 


4 


44 


52 


72 


80 


80 


51 


51 


56 


68 


4 


1 


88 


44 


6 


7 


51 


51 


59 


23 


4 


24 


43 


66 


74 


77 


51 


51 


71 


81 


4 


40 


47 


22 


36 


31 


51 


51 


87 


60 


4 


32 


79 


73 


.84 


87 


51 


51 


92 


61 


5 


18 


68 


52 


4 


4 


51 


51 


1 


1 


5 


57 


34 


35 


69 


85 


51 


51 


5 


18 


5 


30 


21 


16 


100 


97 


51 


51 


17 


36 


5 


23 


54 


47 


87 


84 


51 


51 


24 


55 


5 


70 


76 


46 


55 


51 


51 


51 


26 


4 


5 


80 


85 


64 


28 


34 


51 


51 


28 


. 11 


5 


17 


32 


27 


97 


99 


51 


51 


32 


47 


5 


68 


93 


97 


3 


3 


51 


51 


37 


54 


5 


14 


72 


88 


85 


89 


51 


51 


53 


75 


5 


1 


90 


59 


1 


1 


51 


51 


61 


26 


5 


24 


66 


81 


86 


86 


51 


51 


74 


90 


5 


4 


41 


56 


64 


63 


51 


51 


78 


32 


5 


85 


33 


7 


93 


92 


51 


51 


82 


96 


5 


60 


69 


43 


42 


42 


51 


51 


94 


88 


5 


53 


65 


11 


29 


27 


51 


51 


100 


100 


6 


38 


13 


55 


76 


75 


51 


51 


7 


39 


6 


30 


37 


49 


48 


47 


51 


51 


17 


37 


6 


20 


55 


42 


11 


13 


51 


51 


20 


8 


6 


23 


78 


95 


5 


6 


51 


51 


23 


49 


6 


70 


89 


70 


60 


59 


51 


51 


25 


2 


6 


66 


92 


98 


77 


81 


51 


51 


38 


57 


6 


14 


49 


61 


32 


32 


51 


51 


52 


71 


6 


44 


27 


15 


63 


64 


51 


51 


56 


70 


6 . 


3 


38 


23 


57 


57 


51 


51 


58 


58 


6 


11 


6 


14 


37 


39 


51 


51 


69 


86 


6 


24 


30 


20 


79 


78 


51 


51 


73 


89 


6 


4 


31 


6 


62 


60 


51 


51 


76 


29 


6 


85 


64 


57 


82 


79 


51 


51 


81 


93 


6 


32 


58 


41 


25 


25 


51 


51 


92 


. 69 


6 


60 


59 


32 


71 


66 


51 


51 


94 


87 


6 


53 


95 


85 


12 


11 


51 


51 


98 


98 



20 




23 



Rankings of studies and colleges on 8 performance indicators (continuation)^’ 



study 


cono 
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the table is ordered according to the ranking on gross monthly wages 



Explication: 



study: 2 = pedagogics 

3 = technics 

4 = health studies 

5 = economics 

6 = social studies 

7 = fine arts 



(teacher primary education) 
(mechanical engineering) 
(nurse training) 

(business administration) 
(social work) 

(musician) 



cono: collegenumber 

jeg: ranks based on gross job entry 

jen: ranks based on nett job entry 

jig: ranks based on gross level of job 

jin: ranks based on nett level of job 

whg: ranks based on gross wages per hour 

whn: ranks based on nett wages per hour 

mwg: ranks based on gross montly wages 

mwn: ranks based on nett montly wages 
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